What is Lost in Translation from Visual Graphics to Text for Accessibility

Many blind and low-vision individuals are unable to access digital graphics visually. Currently, the solution to this accessibility problem is to produce text descriptions of visual graphics, which are then translated via text-to-speech screen reader technology. However, if a text description can accurately convey the meaning intended by an author of a visualization, then why did the author create the visualization in the first place? This essay critically examines this problem by comparing the so-called graphic–linguistic distinction to similar distinctions between the properties of sound and speech. It also presents a provisional model for identifying visual properties of graphics that are not conveyed via text-tospeech translations, with the goal of informing the design of more effective sonic translations of visual graphics. Graphics Without Visual Perception Consider the experience of a blind or low-vision individual who uses a screen reader to access pictures, diagrams, charts, and graphs. Unlike a user who accesses graphical media through visual perception, the screen reader user usually accesses these graphics via text-to-speech “descriptions,” essentially interpretations of what was deemed most relevant by the person who produced the text descriptions of the author’s intended meaning. For example, Figure 1a presents a financial chart with rising and falling stock prices over time, where time is shown on the horizontal axis and monetary value is shown on the vertical axis. Figure 1d presents a text description of the chart compliant with the Web Content Accessibility Guidelines (WCAG), using text to describe the rising and falling monetary values over time. The next sections compare and contrast how these presentations are experienced. In a text description of a visual graphic (Figure 1d), all of the information is conveyed via text (or text-to-speech, when conveyed via screen reader technology). But in the original chart (Figure 1a), only some of the information is conveyed via text, predominantly numerical values and labels (Figure 1c); the shape of the shaded contour (Figure 1b) is not conveyed via text: the visually perceived shapes are picked up “more directly” and the features of shapes are translated to text descriptions. However, important properties of visually perceived shape information (Figure 1b) are lost in translation and are instead conveyed via text (Figure 1e). This shape information is needed to provide the unique affordances that are often associated with “visual” representations relative to text. Figure 1. The chart (a) is composed of visually perceived shape contours (b) and text labels (c). Accessibility practices translate b–c to text (d), with shapes described via text (e). Many scholars have explored the differences between graphics and text, often referred to as the so-called “graphic–linguistic distinction” (Shimojima, 1999). In addition, researchers have investigated how so-called “nonlinguistic sonification” can be employed to make charts and graphs more accessible (e.g., Edwards, 2010). This essay examines the graphic–linguistic distinction in order to better understand how it could correspond to a similar distinction between properties of non-linguistic sonification compared to speech to provide a means to identify what is lost when graphics are translated to text-to-speech. An increased understanding could inform the design of new approaches for conveying properties of graphically represented shapes via sound. The Graphic–Linguistic Distinction: Implications for Sonic Interface Design The graphic–linguistic distinction has been described in various ways: analogical versus Fregean; analog versus propositional; graphical versus sentential; and diagrammatical versus linguistic (Shimojima, 1999). 1 Adapted from “Web Accessibility Best Practices: Graphs” by Campus Information Technologies and Educational Services (CITES) and Disability Resources and Educational Services (DRES), University of Illinois at Urbana/Champaign. Copyright 2005 by University of Illinois at Urbana/Champaign.